1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method and an image processing program which can detect a direction of an edge in the vicinity of a pixel of interest based on a number of color components, and a recording medium recording the image processing program.
For example, the present invention relates to an imaging apparatus, such as a digital camera, a camcorder, a mobile telephone with camera or the like, which employs a single solid-state imaging device, such as a CCD, a CMOS or the like. More particularly, the present invention relates to an image processing apparatus, an image processing method and an image processing program which generate all color signals for each pixel by interpolation using color signals of a pixel of interest and its neighbor pixels based on color signals output from a solid-state imaging device in which a plurality of color filters are individually disposed on respective pixels. The present invention also relates to a recording medium recording the image processing program.
2. Description of the Related Art
In general, solid-state imaging devices (e.g., a CCD and a CMOS for use in digital cameras or camcorders) detect only information indicating the brightness of light. Therefore, in order for a camera having a single solid-state imaging device (e.g., CCD) to obtain color information, pixels contained in the CCD are individually covered with respective color filters each of which passes only one color component, so that each pixel outputs a corresponding color component during operation. When the color filters are primary color filters, there are three color components: red (R), green (G) and blue (B), for example.
Several types of arrangement of color components of a color filter have been proposed.
FIG. 15 shows a Bayer arrangement, where a checker board array of G pixels which contribute to a large proportion of a brightness signal is patterned, and half of the remaining spaces are then provided with R pixels and the remainder with B pixels. At a time when a signal is output from a CCD covered with the Bayer arrangement color filter, information about only one color of the RGB color components is obtained from each pixel.
Therefore, an image processing section estimates information about the remaining two colors for each pixel by interpolation using color signal values of its surrounding pixels. Further, the one-color information obtained by the CCD for each pixel is also subjected to interpolation using color signal values of the pixel of interest and its surrounding pixels.
FIGS. 16A to 16C show Bayer arrangements in which a pixel of interest has a color component of G, R or B.
FIG. 16A shows a Bayer arrangement in which a pixel of interest has a color component of G. FIG. 16B shows a Bayer arrangement in which a pixel of interest has a color component of R or B. FIG. 16C shows a Bayer arrangement in which a pixel of interest has an arbitrary color component.
In FIGS. 16A and 16B, symbols P and Q indicate respectively the color component R and the color component B, or the color component B and the color component R. When the symbol P indicates the color component R, the symbol Q indicates the color component B. Conversely, when the symbol P indicates the color component B, the symbol Q indicates the color component R.
An interpolation expression (Expression 1) according to a bilinear method where a color filter has the Bayer arrangement will be described as follows.
(Expression 1)
When the pixel of interest is G33 (see FIG. 16A),Go=G33 or Go=(G33*4+G22+G42+G24+G44)/8,Po=(P32+P34)/2, andQo=(Q23+Q43)/2.
When the pixel of interest is P33 (see FIG. 16B),Go=(G32+G23+G43+G34)/4,Po=P33, andQo=(Q22+Q42+Q24+Q44)/4,    Go: output G signal of the pixel of interest,    Po: output P signal of the pixel of interest,    Qo: output Q signal of the pixel of interest, and note that (P, Q)=(R, B) or (B, R).
Since the bilinear method is a kind of lowpass process, it is advantageous that noise is caused not to be noticeable. However, a high frequency component in spatial frequencies of an object, such as an edge of the object or the like, is lost, disadvantageously leading to a reduction in resolution.
Therefore, a technique of performing interpolation by investigating correlation in signal value between a pixel of interest and its neighbor pixels using image data having the Bayer arrangement, has been proposed (e.g., Japanese Laid-Open Publication No. 10-164602 (Patent Publication 1), Japanese Laid-Open Publication No. 11-122626 (Patent Publication 2), Japanese Laid-Open Publication No. 2000-165892 (Patent Publication 3), and Japanese Laid-Open Publication No. 2003-143615 (Patent Publication 4)). In this interpolation process, the presence or absence of an edge in the vicinity of the pixel of interest and a directionality of the edge are detected, a color signal of a pixel having a high level of correlation in the detected edge direction is given a greater weight in order to prevent loss of a high frequency component indicating an edge in the image data.
[Patent Publication 1] Japanese Laid-Open Publication No. 10-164602
[Patent Publication 2] Japanese Laid-Open Publication No. 11-122626
[Patent Publication 3] Japanese Laid-Open Publication No. 2000-165892
[Patent Publication 4] Japanese Laid-Open Publication No. 2003-143615
However, in the techniques disclosed in Patent Publication 1, Patent Publication 2 and Patent Publication 3, whereas G pixels the number of which is largest in the RGB color components and which contribute to a large proportion of a brightness signal are used in edge detection, R pixels and B pixels which share half of image data are not used. Therefore, when image data has less G components, the accuracy of edge detection is deteriorated.
On the other hand, in the technique disclosed in Patent Publication 4, when the color element of a pixel of interest is an R pixel, a G pixel and an R pixel are used for edge detection. When the color element of a pixel of interest is a B pixel, a G pixel and a B pixel are used for edge detection. However, when the color element of a pixel of interest is a G pixel, only a G pixel is used. Therefore, when image data has a less number of G components, the accuracy of edge detection is deteriorated.
Further, in the techniques disclosed in Patent Publications 1 to 4, when image data contains noise, an influence of the noise may lead to an error in edge detection.